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Abstract. In this paper wc pursue the study of formal geometric quantization 
of non-compact Hamiltonian manifolds. Our main result is the proof that two 
quantization process coincide. This fact was obtained by Ma and Zhang in the 
preprint arXiv:0812.3989 by completely different means. 
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In the previous article [21] , we have studied some functorial properties of the 
"formal geometric quantization" process Q~°°, which is defined on proper Hamil- 
tonian manifolds, e.g. non-compact Hamiltonian manifolds with proper moment 
map. 

There is another way, denoted Q*, of quantizing proper Hamiltonian manifolds 
by localizing the index of the Dolbeault Dirac operator on the critical points of the 
square of the moment map [15l H9l [20] . 

The main purpose of this paper is to provide a geometric proof that the quanti- 
zation process Q~°° and Q* coincide. This fact was proved by Ma and Zhang in 
the recent preprint [15] by completely different means. 

Keywords: moment map ; symplcctic reduction ; geometric quantization ; transver- 
sally elliptic symbol. 



1. Introduction and statement of results 

Let us first recall the definition of the geometric quantization of a smooth and 
compact Hamiltonian manifold. Then wc show two way of extending the notion of 
geometric quantization to the case of a non-compact Hamiltonian manifold. 

Let if be a compact connected Lie group, with Lie algebra t. In the Kostant- 
Souriau framework, a Hamiltonian if-manifold (M, f2, $) is pre-quantized if there 
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is an equivariant Hcrmitian line bundle L with an invariant Hermitian connection 
V such that 

(1.1) C(X) - Vx M = X ) and V 2 = -ifi, 

for every I £ t. Here Xm is the vector field on M defined by Xm{tti) = -^e~ tx m\o- 
The data (L, V) is also called a Kostant-Souriau line bundle, and <!> : M — ► 6* is 
the moment map. Remark that conditions (|1.1[) imply via the equivariant Bianchi 
formula the relation 

(1.2) L(X M )Sl = -d($,X), Xet. 

Let us recall the notion of geometric quantization when M is compact. Choose 
a A-invariant almost complex structure J on M which is compatible with Q in 
the sense that the symmetric bilinear form f2(-, J-) is a Riemannian metric. Let 
Ol be the Dolbeault operator with coefficients in A, and let d L be its (formal) 
adjoint. The Dolbeault- Dime operator on M with coefficients in L is Dj, = + 
considered as an elliptic operator from A°> cvcn {M,L) to A°' odd {M, L). Let R(K) 
be the representation ring of K. 

Definition 1.1. The geometric quantization of a compact Hamiltonian K -manifold 
(M, f2, $) is the element Qk{M) G defined as the equivariant index of the 

Dolbeault- Dirac operator D^. 

Let us consider the case of a proper Hamiltonian A-manifold M: the manifold 
is (perhaps) non-compact but the moment map $ : M — > 6* is supposed to 
be proper. Under this properness assumption, one define the formal geometric 
quantization of M as an element Q K °°(M) that belongs to R~°°(K) [3Q1EI]. Let 
us recall the definition. 

Let T be a maximal torus of K. Let t* be the dual of the Lie algebra of T 
containing the weight lattice A* : a 6 A* if ia : t — > iR is the differential of a 
character of T. Let Ck C t* be a Weyl chamber, and let K := A* n Ck be the set 
of dominant weights. The ring of characters R(K) has a Z-basis V^,fi £ K : Vff 
is the irreducible representation of K with highest weight ix. 

A representation £ of if is admissible if it has finite A"-multiplicities : 
dim(hom, R :(V A f , E)) < oo for every \i G K. Let R-°°(K) be the Grothendieck 
group associated to the AT-admissible representations. We have an inclusion map 
R{K) ^ R-°°(K) and R-°°(K) is canonically identify with hom z (i?(A), Z). 

For any fi € K which is a regular value of moment map <f>, the reduced space (or 
symplectic quotient) := $ _1 (A' • n)/K is a compact orbifold equipped with a 
symplectic structure f2 M . Moreover := (A|$-i( A1 )(K'C_ A1 )/A' Al is a Kostant-Souriau 
line orbibundlc over (M M ,f2 M ). The definition of the index of the Dolbeault-Dirac 
operator carries over to the orbifold case, hence Q(M^) G Z is defined. In Section 
12 .31 we explain how this notion of geometric quantization extends further to the 
case of singular symplectic quotients. So the integer Q(M^) £ Z is well defined for 
every fi£i(: in particular Q(M M ) = if /i ^ 3>(M). 

Definition 1.2. Lei (M, 17,$) &e a proper Hamiltonian K-manifold which is pre- 
quantized by a Kostant-Souriau line bundle L. The formal quantization of (M, ft, <£>) 
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is the element of R °°{K) defined by 

When M is compact, the fact that 

(1.3) Qk(M) = Qk°°{M) 

is known as the "quantization commutes with reduction Theorem" . This was con- 
jectured by Guillcmin- Sternberg in [9] and was first proved by Meinrenken |17j and 
Meinrenken-Sjamaar [18] ■ Other proofs of (jl.3p were also given by Tian-Zhang 
[26] and the author [19]. For complete references on the subject the reader should 
consult [25]. [28]. 

One of the main feature of the formal geometric quantization Q~°° is its stability 
relatively to the restriction to subgroups. 

Theorem 1.3 (|21j). Let M be a pre- quantized Hamiltonian K -manifold which 
is proper. Let H C K be a closed connected Lie subgroup such that M is still 
proper as a Hamiltonian H -manifold. Then Q^°°(M) is H -admissible and we 
have Qk°°(M)\ h = Qh°°(M) m R~°°(H). 

When M is a proper Hamiltonian A'-manifold, we can also define another "formal 
geometric quantization" , denoted 

(1.4) Q%{M) G R-°°(K), 

by localizing the index of the Dolbcault-Dirac operator Dl on the set Cr(||$|| 2 ) 
of critical points of the square of the moment map (see Section 12.21 for the precise 
definiton) . We proved in previous papers [20] [21] [23] that 

(1.5) Q K °°(M) = Q%(M). 
in somes situations: 

• M is a coadjoint orbit of a semi-simple Lie group S that parametrizes a rep- 
resentation of the discrete series of S, 

• M is a Hermitian vector space. 

In her ICM 2006 plenary lecture [29], Vergne conjectured that (|1.5p holds when 
Cr(||$|| 2 ) is compact. Recently, Ma and Zhang [15] prove the following generalisa- 
tion of this conjecture. 

Theorem 1.4. The equality fl.5\) holds for any proper Hamiltonian K -manifold. 

This article is dedicated to the study of the quantization map Q*: 

• In Section ^. 2i we give the precise definition of the quantization process Q* . 
In particular, we refine the constant c 7 appearing in |15] [Theorem 0.1]. 

• In Section [2. A\ we explain how to compute the quantization of a point. 

• We give in Section [3] another proof of Theorem 11.41 by using the technique 
of symplectic cutting developped in [21j . 

• In Section [4] we consider the case where K = K\ x if 2 acts on M in a way 
that the symplectic reduction M // qK\ is a smooth proper A"2-Hamiltonian 
manifold. We show then that the A^i -invariant part of <2|- xK (M) is equal 

to Q^(A/// Ai). 
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In Section [3 we study the example where M is the cotangent bundle of a ho- 
mogeneous space: M = T*(K/H) where H is a closed subgroup of K. We see 
that T*{K/H) is a proper Hamiltonian _ftT-manifold prequantizcd by the trivial line 
bundle. A direct computation gives 

(1.6) Q%{T*(K/H)) = L 2 (K/H) in R~°°(K). 

Let us denoted \T*(K / H)]^^ the symplectic reduction at \i € K of the K- 
Hamiltonian manifold T*(K/H). Theorem II .41 together with (|1.6p give 

Q{[T*(K/H)] ll , K ) = dim[V?] H , 

for any [i G K. Here [V^} H C V^f is the subspace of H -invariant vectors. 

Then we consider the action of a closed connected subgroup G C K on T*(K/H). 
We first check that T*(K/H) is a proper Hamiltonian G-manifold if and only if the 
restriction L 2 (K/H)\g is an admissible G-representation. Then, using Theorem 
11.31 we get that 

(1.7) Q G °°(T*(K/H))=L 2 (K/H)\ G in iT°°(G). 

In other words, the multiplicity of in L 2 (K/H) is equal to the quantization of 
the reduced space [T*(K/H)] x ,g- 

2. Quantizations of non-compact manifolds 

In this section wc define the quantization process Q* , and we give another defi- 
nition of the quantization process Q~°° that uses the notion of symplectic cutting 

ED- 

2.1. Transversally elliptic symbols. Here we give the basic definitions from the 
theory of transversally elliptic symbols (or operators) defined by Atiyah-Singer in 
[Tj . For an axiomatic treatment of the index morphism see Berline-Vergne [7] 
and Paradan-Vcrgne [22] . For a short introduction see [TI5] . 

Let X be a compact i^-manifold. Let p : TX — > X be the projection, and let 
(-,-)^ be a JsT-invariant Ricmannian metric. If E°, E 1 are i£f-equivariant complex 
vector bundles over X, a X-cquivariant morphism a G T(TX, hom(p* E° ,p* E 1 )) is 
called a symbol on X. The subset of all (x,v) G TX whcrcQ a{x,v) : — > E\ is 
not invertible is called the characteristic set of a, and is denoted by Char(er). 

In the following, the product of a symbol a by a complex vector bundle F — > M, 
is the symbol 

a® F 

defined by a ® F(x,v) = a(x,v) <g> Id F:c from S° <g> F s to £7^ (g) F x . Note that 
Char(cr® f 1 ) = Char(cr). 

Let TkX be the following subset of TX : 

T K X = {{x, v) G TX, {v,X x {x)) x =0 forallXGt}. 

A symbol a is elliptic if cr is invertible outside a compact subset of TX (i.e. 
Char(cr) is compact), and is K-transversally elliptic if the restriction of a to TkX 
is invertible outside a compact subset of TkX (i.e. Char(cr) n Tk 2 X is compact). 
An elliptic symbol a defines an element in the equivariant K-theory of TX with 
compact support, which is denoted by Hk{TX), and the index of a is a virtual finite 
dimensional representation of K, that we denote Index^(a) G R{K) [2J [31 SI Ej - 



The map cr(x, v) will be also denote a\ x {v) 
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Let 

R£°(K) c R-°°(K) 
be the R(K )-submodule formed by all the infinite sum YIu^k where the map 

H £ K i—t rrin £ Z has at most a polynomial growth. The i?(i£T)-module R^ C °°(K) is 
the Grothendieck group associated to the trace class virtual if-representations: we 
can associate to any V <E R^ C °°(K), its trace k — ► Tr(fc, V) which is a generalized 
function on K invariant by conjugation. Then the trace defines a morphism of 
R(K)-module 

(2.8) RTc°°( K ) ^ C-°°{K) K . 

A K-transversally elliptic symbol a defines an element of K^(T^A'), and the 
index of a is defined as a trace class virtual representation of K , that we still denote 
Index^((j) G R^°{K). 

Remark that any elliptic symbol of TX is if-transversally elliptic, hence wc have 
a restriction map K^(TA') — > Kx(TjfA'), and a commutative diagram 

(2.9) K K (TX) K K (T K X) 



Index*; 



Index* 



R{K) >R£°{K) . 

Using the excision property, one can easily show that the index map Index^ : 
Kk(T^W) — > R^^°(K) is still defined when U is a -invariant relatively compact 
open subset of a K- manifold (see [TH] [section 3.1]). 

Suppose now that the group K is equal to the product K\ x K 2 . When a symbol 
a is K\ x i^-transversaly elliptic we will be interested in the -invariant part of 
its index, that we denote 

Index^ lxX2 ( ( 7)] Al e R7 C °°(K 2 ). 

An intermediate notion between the "ellipticity" and U K\ x ^-transversal ellip- 
ticity" is the "ifi-transversal ellipticity" . When a K\ x i^-cquivariant morphism 
a is Ki-transver sally elliptic, its index Index^ 1 x K2 (a) € R^ c °°(Ki x K%), viewed 
as a generalized function on K\ x K 2 , is smooth relatively to the variable in K 2 . It 
implies that Index^ 1 x K2 (a) = £ A 6»(A) ® v \' with 

0(A) e R(K 2 ), V A e K\. 

In particular, we know that 

Indexf^V)]* 1 =9(0) 
belongs to R(K 2 ). 

Let us recall the multiplicative property of the index map for the product of 
manifolds that was proved by Atiyah-Singer in pQ. Consider a compact Lie group 
K 2 acting on two manifolds X\ and X 2 , and assume that another compact Lie group 
K\ acts on X\ commuting with the action of K 2 . 

The external product of complexes on TX\ and TA2 induces a multiplication 
(see [D[22]): 

© : KK 1 xK 2 ( r ^K 1 Xi) x Kk^KiX-i) — > Kk 1 xk 2 (Tk 1 xk 2 {Xi x %2))- 
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The following property will be used frequently in the paper. 

Theorem 2.1 (Multiplicative property). For any [01] € Kk 1 xk 2 {'^k 1 '^i) and any 
[a 2 ] G Kk 2 (Tk 2 X2) we have 

Index£ x *f (H [aa]) = Index£ x ^(M) Index£([<r 2 ]). 

We will use in this article the notion of support of a generalized character. 

Definition 2.2. TTie support of x ■= EuK<fy^f e R^°°(K) is the set of fx £ A' 
smc/i £/ia£ a M =/= 0. 

We will say that x G R~°°{K) is supported outside £> C t* if the support of x 
does not intersect B. Note that an infinite sum Y^ieiXi converges in A~°°(A) if 
for each ball 

B r = u e e | neii < r} 

the set {i G 7 | support(xi) n B r ^ 0} is finite. 

Definition 2.3. VFe denote by 0(r) any character of R~°°(K) which is supported 
outside the ball B r . 

2.2. Definition and first properties of Q*. Let (M, fi,$) be a proper Hamil- 
tonian A- manifold prcquantized by an cquivariant line bundle L. Let J be an 
invariant almost complex structure compatible with £1. Let p : TM — > M be the 
projection. 

Let us first describe the principal symbol of the Dolbcault-Dirac operator 8l + 
d L . The complex vector bundle (T*!/) - 1 is A'-cquivariantly identified with the 
tangent bundle TM equipped with the complex structure J. Let h be the Hermitian 
structure on (TM, J) defined by : h(v,w) = Cl(v,Jw) — iQ(v,w) for v, w G TM. 
The symbol 

Thom(M, J) G L (A/, hom(p*(Ag ue,l TM), /(A^TAf))) 
at (m, v) £ TM is equal to the Clifford map 

(2.10) c m (v) : A e c ven T m M A° c dd T m M, 

where c m (t>).w = vf\w-i(v)w for w G A*T m AA Here l(v) : A*T m M — > A'~ 1 T m M 
denotes the contraction map relative to h. Since c m (v) 2 = — ||w|| 2 Id. the map c m (v) 
is invertible for all v ^ 0. Hence the characteristic set of Thom(M, J) corresponds 
to the 0-section of TM. 

It is a classical fact that the principal symbol of the Dolbeault-Dirac operator 
8l + d* L is equal tc0 

(2.11) Thorn (A/, J)® L, 

see 0. Here also we have Char(Thom(M, J) ® L) = - section of TM. 

Remark 2.4. When the manifold M is a product M\X M% the symbol Thom(M, J)0 
L is equal to the product o\ o~i where o~k = Thom(Mfc, Jk) 7/c- 



Here we use an identification T*M ~ TM given by an invariant Ricmannian metric. 
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When M is compact, the symbol Thom(M, J) (g) L is elliptic and then defines an 
element of the equivariant K-group of TM. The topological index of Thom(M, J)(g) 
L G K.k (TM) is equal to the analytical index of the Dolbeault-Dirac operator 
d L + d* L : 

(2.12) Qk(M) = Index^(Thom(M, J) <g> L) in R(K). 

When M is not compact the topological index of Thom(M, J) <3>L is not defined. 
In order to extend the notion of geometric quantization to this setting we deform the 
symbol Thom(M, J) ® L in the "Witten" way [IS HO]. Consider the identification 
^ i — > ^, fi* — ^ £ defined by a if-invariant scalar product on t* . We define the Kirwan 
vector field on M : 

(2.13) K m = ( $(m) J (m), m G M. 

Definition 2.5. TTie symbol Thom(M, J) ® L pushed by the vector field k is the 
symbol c K defined by the relation 

c K \ m (v) = Thom(M, J) (g> L\ m (v - n m ) 

for any (m, v) G TM . 

Note that c K \ m (v) is invertible except if v = K m . If furthermore v belongs to 
the subset TjfM of tangent vectors orthogonal to the if -or bits, then v = and 
K m = 0. Indeed K m is tangent to K ■ m while v is orthogonal. 

Since n is the Hamiltonian vector field of the function ^.||$|| 2 , the set of zeros 
of k coincides with the set Cr(||$|| 2 ) of critical points of ||$|| 2 . Finally we have 

Char(c K )nT x M~Cr(||$|| 2 ). 

In general Cr(||$|| 2 ) is not compact, so c K does not define a transversally elliptic 
symbol on M. In order to define a kind of index of c K , we proceed as follows. For 
any invariant open relatively compact subset U C M the set Char(c K |[/) n TkU ~ 
Cr(||$|| 2 ) n U is compact when 

(2.14) aWnCr(||$|| 2 ) = 0. 
When (|2.14|) holds we denote 

(2.15) Q*(U):= Index* (c K \u) G R^(K) 

the equivariant index of the transversally elliptic symbol c K \u. 

It will be uscfull to understand the dependance of the generalized character 
Q%(U) relatively to the data (U, 0, L). So we consider two proper Hamilonian K- 
manifolds (M, f2, <I>) and (M 1 , 0', $') respectively prequantized by the line bundles 
L and L'. Let V C M and V C M' two invariant open subsets. 

Proposition 2.6. • The generalized character Q* (U) does not depend of the choice 
of an invariant almost complex structure on U which is compatible with Q\u- 

• Suppose that there exists an equivariant diffeomorphism $ : V — * V' such that 

(1) **($') = 

(2) **(£') = L, 

(3) there exists an homotopy of symplectic forms taking ^ r *(f2'|y) to f2|v- 
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Let U' C U' C V' &e an invariant open relatively compact subset such that dU' 
satisfies [21$ . Take U = ^(U'). Then dU satisfies $TTJ\ ) and 

Q%(U') = Q%(U) G R-°°(K). 

Proof. Let us prove the first point. Let c?|y,i = 0,1 be the transversally elliptic 
symbols defined with the compatible almost complex structure Ji,i = 0,1. Since 
the space of compatible almost complex structure is contractible, there exist an 
homotopy J t ,t e [0,1] of almost complex structures linking J and J\. If we 
use Lemma 2.2 in |19j . we know that there exists an invertible bundle map A £ 
F([7, End(TC/)), homotopic to the identity, such that A o J = J x o A. With the 
help of A we prove then that the symbols Cq\u and cf|[/ define the same class in 
K if (Tiff/) (see [19] [Lemma 2.2]). Hence their equivariant index coincide. 

Let us prove the second point. The characters Q%(U) and Q|- (U 1 ) arc computed 
as the equivariant index of the symbols c K \u and c K \w- Let c K \u the pull back 
of c K \xji by "J. Thanks to the point (1) and (2), the only thing which differs in 
the definitions of the symbols c K \u and c K \u are the almost complex structures 
J and J = iff* (J') : the first one is comptible with fl and the second one with 
^*(Q'\v)- Since these two symplectic structure are homotopic, one sees that the 
almost complex structures J and J are also homotopic. So wc can conclude like in 
the first point. □ 

Let us recall the basic fact concerning the singular values of ||$|| 2 . 

Lemma 2.7. The set of singular values of \\&\\ 2 : AI —> M forms a sequence 
< n < ^2 < . . . < rfe < . . . which is finite iff Cr(||$|| 2 ) is compact. In the other 
case limfc^oo = oo. 

At each regular value R of Cr(||$|| 2 ), wc associate the invariant open subset 
M <R := {|| $11 2 < R] which satisfies (|2.14[) . The restriction c K \ M<R defines then a 
transversally elliptic symbol on M<ij: let Q% : (M < r) be its equivariant index. 

Let us show that Q%(M < n) has a limit when R — > oo. The set Cr(||$|j 2 ) has 
the following decomposition 

(2.16) Cr(||$|| 2 ) = (J K- (Af^n*- 1 ^)) 

/36B % J ' 

where the B is a subset of the Weyl chamber ti_. Note that each part Zp is compact, 
hence B is finite only if Cr(||<I>|j 2 ) is compact. When Cr(|j<i>|| 2 ) is non-compact, the 
set B is infinite, but it is easy to see that B H B r is finite for any r > 0. For any 
(3 € B, we consider a relatively compact open invariant neighborhood hip of Zp such 
that Cr(||$|| 2 )n^ = Z /3 . 

Definition 2.8. We denote 

QP K (M) e Rt c °°(K) 
the inde^ of the transversally elliptic symbol c K \ua- 

A simple application of the excision property [TH] gives that 

(2-17) Q%( M<R ):= J2 QkW- 

W¥<R 

3 The index of c K | M/3 was denoted RR^ (M, L) in [19]. 
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We have now the key fact 

Theorem 2.9. The generalized character Q K {M) is supported outside the open 
ball -B||/3|| • 



Proof. Proposition 12.91 follows directly from the computations done in [19J. First 
consider the case where (3 ^ is a A"-invariant element of B. Let i : Tp T 
be the compact torus generated by (3. If F is Z-module we denote F<%> i? -00 (Tg) 
the Z-module formed by the infinite formal sums ^ a E a h a taken over the set of 
weights of Tg, where E a G F for every a. 

Since Tg lies in the center of K, the morphism ir : (k, t) E K x kt G K 

induces a map n* : R-°°(K) -»• R-°°(K) § R-°°(T fi ). 

The normal bundle AT of M° in M inherits a canonical complex structure J/v 
on the fibers. We denote by Af — > the complex vector bundle with the opposite 
complex structure. The torus is included in the center of K, so the bundle J\f 

and the virtual bundle A* AT := A™ e? W — > A^f d 77 carry a if x T^-action: they can 
be considered as elements of A KxT/3 (M^) = K K (M^) ® R(T P ). 

In [19], we have defined an inverse of AJA/", [aJJV"]^ 1 G K k {M^) ® R-°°(Tp), 

which is polarized by (3. It means that [a'A/"]^ 1 = ^2 a N a h a with N a ^ only if 

(a,(3)>0. 

We prove in []Ij] the following localization formula : 
(2.18) n* [fi&(M)] = RR^ Jfi (m$, L\ M e <g> [aJJNT]" 1 



as an equality in R~°°(K) §> R~°°(Tp). With (j2~18]) in hand, it is easy to see that 
V* occurs in the character Q? K {M) only if > ||/3|| 2 (See Lemma 9.4 in [19]). 

Now we consider the case were (3 G B is not a A-invariant element. Let cr be 
the unique open face of the Weyl chamber t+ which contains f3. Let K a be the 
corresponding stabilizer subgroup. We consider the symplectic slice y a C M: it is 
a K a invariant Hamiltonian submanifold of M which is prequantized by the line 
bundle L\y a . The restriction of $ to 3^<r is a moment map § a : y a — > 6* which is 
proper in a neighborhood of j3 G 6* . The set 

^•(^n$; 1 (/3)) = M^n$" 1 (/3) 

is a component of Cr(||$o-|| 2 ). Let Q^- (3^.) G R^ c °°(K a ) be the corresponding 
character (see Definition ^. 8|l . 

We prove in |19] [Section 7] , the following induction formula: 

(2.i9) e&(M) = Ho£ (e^(y«r) 

where Hol^ : R -00 ^^) — > R~°°(K) is the holomorphic induction map. See the 
Appendix in [19] for the definition and properties of these induction maps. 
We know from the previous case that 
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where to m ^ => (//,/?) > \\/3\\ 2 . Then, with ([2TT9]) . we get 

(M,/3)>ll/9|| 2 

£ m„Hb£(f), 

(^,/3)>lf/3|| 2 

where Hol T : i? _oc '(T) — > R~°°(K) is the holomorphic induction map. 

Let p be half the sum of the positive roots. The term Hol T is equal to 
when p + p is not a regular element of t* . When p + p is a regular element of t* , 
we have Hoi* (f) = (-1) H V^ where 

/x w = uj(p + p) - p 

is dominant for a unique u € W. 

Finally, a representation VP appears in the character Q^ K {M) only if A = fj, u 
for a weight p satisfying (p, (3) > ||/3|| 2 . Hence, for such A, we have 

j|A|| = \\p + p - oj^ 1 p\\ 

> (M + P---V,^) 

> 11/311- 

In the last inequality we use that (p — uj^ 1 p, (3) > since p — oj~ 1 p is a sum of 
positive roots, and /3 E t*,. 

□ 

With the help of Theorem 12.91 and decomposition (|2.17j) , we see that the multi- 
plicity of in Q%{M < r) does not depend on the regular value R > \\j\\ 2 . We 
can refine the constant c 7 appearing in [15 [Theorem 0.1]: take c 7 equal to ||7|| 2 
instead og || 7 + p\\ 2 - \\p\\ 2 > || 7 || 2 . 

Definition 2.10. The generalized character Q%{M) is defined as the limit in 
R~°°(K) of Q%(M < r) when R goes to infinity. In other words 

(2.20) Q|(M) = J2 Qk(M). 

Note that for any regular value R of ||$|| 2 we have the useful relation 

(2.21) Q|(M) = Q%(M <R ) + 0(VR). 

2.3. Quantization of a symplectic quotient. We will now explain how we de- 
fine the geometric quantization of singular compact Hamiltonian manifolds : here 
"singular" means that the manifold is obtained by symplectic reduction. 

Let (TV, fi) be a smooth symplectic manifold equipped with a Hamiltonian action 
of K\ x K2 : we denote ($1,^2) : N —> $* X t% the corresponding moment map. 
We assume that TV is pre-quantized by a K\ x i^-equivariant line bundle L and we 



^Here p is half the sum of the positive roots. Hence H7 + p|| 2 — ||p|| 2 — ||7|| 2 = 2(p, 7) > and 
(p, 7) = only if the weight 7 belongs to the center of £ ~ t* . 
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suppose that the map $1 is proper. One wants to define the geometric quantization 
of the (compact) symplectic quotient 

N//^ :=$ r 1 (0)/if 1 . 

Let Ki be the Kirwan vector field attached to the moment map <£>i. We denote 
by c Kl the symbol Thom(./V, J) <S> L pushed by the vector field K\. For any regular 
value Ri of ||$i|| 2 , we consider the restriction c Ki |at <Bi to the invariant, open 
subset N < n 1 := {H'J'ill 2 < Ri}- The symbol c Ki \n <Ri is K x x if 2 -equivariant and 
i^i-transversally elliptic, hence we can consider its index 

Index£*f 2 (c«MiV <Rl ) G BT^{K X X K 2 ). 

which is smooth relatively to the parameter in K 2 . We consider the following 
extension of Definition 12.101 

Definition 2.11. The generalized character Q% 1 iX k 2 ^) * s defined as the limit in 
R-° a (K 1 x K 2 ) o/Index^^ 2 (c Kl |jv <Rl ) when R 1 goes to infinity. 

Here Cr(||<3>i|j 2 ) is equal to the disjoint union of the compact K\ x ^-invariant 
subsets Zfo := K x ■ (M ft n ^(A)), ft e B x . For ft G B lt we consider an 
invariant relatively compact open subset Uf3 1 such that: C and Zp t = 

Cr(||$i|| 2 ) DU^. Let Qk ixK ^(N) G R-°°(K 1 x K 2 ) be the cquivariant index 
of the ifi-transversally elliptic symbol c^V \u l3l ■ The JsTi-transversallity condition 
imposes that Q^ lXK2 (N) = £ A 6> ft (A) ® v \' with 

O^iX) G R{K 2 ), VAeFx. 
We have the following extension of Theorem 12.91 

Theorem 2.12. We have Qk iX k 2 ( N ) = J2xeKl 901 ( X ) ® V \ % where 901 W ^ 
onlyif\\\\\ > ||ft || . 

Proof. The proof works exactly like the one of Theorem 12.91 □ 

Let us explain the "quantization commutes with reduction theorem" , or why we 
can consider the geometric quantization of 

N// Q K X := $^(0)/*! 

as the i^i-invariant part of Qk iXK (N). 

Let us first suppose that is a regular value of $i. Then N// K\ is a compact 
symplectic orbifold equipped with a Hamiltonian action of K 2 : the corresponding 
moment map is induced by the restriction of $2 to < I ) ^ 1 (0). The symplectic quotient 
N // qK\ is prc-quantized by the line orbibundlc 

L o ■= ( L Ur>)) 

Definition 11.11 extends to the orbifold case. We can still define the geometric 
quantization of N // Q K\ as the index of an elliptic operator : we denote it by 
Qk 2 (N// q Kx) G R(K 2 ). Wc have 

Theorem 2.13. IfO is a regular value o/$ l7 the Ki-invariant part of xK {N) 
is equal to Qk 2 (N// K 1 ) G R{K 2 ). 
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Suppose now that is not a regular value of $1. Let T\ be a maximal torus of 
Kx, and let C\ C i\ be a Weyl chamber. Since <l>i is proper, the convexity Theorem 
says that the image of $1 intersects C\ in a closed locally polyhedral convex set, 
that we denote A Kl (AT) [14] . 

We consider an element a G (iV) which is generic and sufficiently close to 
G Ajfj (JV) : we denote (-Ki) a the subgroup of K\ which stabilizes a. When 
a G Aif 1 (A r ) is generic, one can show (see [18]) that 

Nff a Kx :=*£(a)/(#i) a 
is a compact Hamiltonian A^-orbifold, and that 



/(*l)- 



is a A^-equivariant line orbibundle over N jj a K\ : we can then define, like in Def- 
inition [TTT] the clement Qk 2 {N // a K\) G R{K2) as the equivariant index of the 
Dolbcault-Dirac operator on N // K\ (with coefficients in L a ). 



Theorem 2.14. 



The K\-invariant part of Q* 1 



K 1 xK : 



(M) is equal to Qk 2 (N '// ' a K x ) G 
R(K2). In particular, the elements Qk 2 {N // Ki) do not depend on the choice of 
the generic element a € Ah(N), when a is sufficiently close to 0. 



Proofs of Theorem 12.131 an Theorem 12.141 . When N is compact and 
K-2 = {e}, the proofs can be found in [TS] and in [TH]. Let us explain briefly how 
the K-theoretic proof of [19] extends naturally to our case. Like in Definition ^. 101 
we have the following decomposition 



Q 



if! X if 2 



(AO 



E 

/3SBi 



Q 



0i 

KixK 2 



(N), 



And Theorem 12.121 tells us that 
the first step: 



Q 



Pi 

KixK 2 



(N) 



Ki 



if j3\ ^ 0. We have proved 



Q%\xk 2 (N) 



Ki 



[Q°KixK 2 (N)] 



Ki 



The analysis of the term [Q% 1 xK (Af)] Kl is undertaken in [TH] when K2 = {e}: 
we explain that this term is equal either to Q(N // K\) when is a regular value, 
or to Q(N jj a K\) with a generic. It work similarly with an action of a compact Lie 
group Ki- □ 

Definition 2.15. The geometric quantization of N // Q Ki := (0)/Ki is taken 
as the Ki-invariant part of Q% 1 iX k 2 (N)- We denote it Qk 2 (N// Ki). 



2.4. Quantization of points. Let (M, f2, <£) be a proper Hamiltonian A'-manifold 
prequantized by a Kostant-Souriau line bundle L. Let \x G K be dominant weight 
such that §~ X (K • fj,) is a AT-orbit in M. Let m° G 3> _1 (/i) so that. 

$ _1 (AT ■ fi) =K-m° 

Then the reduced space M M := $>~ 1 (K • /j,)/K is a point. The aim of this section is 
to compute the quantization of M M : Q(M^) G Z. 

Let -ff be the stabilizer subgroup of m°. We have a linear action of H on the 
1-dimensional vector space L m ° C L. We have H C A" M where is the connected 
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subgroup of K that fixes [i £ t*. Let C-^ be the 1-dimensional representation of 
associated to the infinitesimal character — iji. 
Let us denote x be the character of H defined by the 1-dimensional representation 
C x := L m o <gi C_ M . We know from the Kostant formula (jl.ip that \ = 1 on the 
identity component H° C H. 



Theorem 2.16. We have 
(2.22) Q(M p 



1 if x = 1 on H 
in the other case. 



This Theorem tells us in particular that Q(M M ) = 1 when the stabiliser subgroup 
H C K of a point m° £ $ -1 (/z) is connected. 



Proof. Let N = M X K ■ /j, be the proper Hamiltonian A-manifold which is pre- 
quantized by the line bundle Ljy := L ® [C_^]. Let us denote the moment 
map on N . Since <1> _1 (A" • fi) is a X-orbit in M, we see that $^(0) is the if-orbit 
through n° (m°, fi) where m° £ $ _1 (/i). Note that H is the stabilizer subgroups 
of n°. 

Let Q^ N (N) £ R~°°(K) be the formal quantization of N through the proper 
map <I>jv. By definition 

e(M„) = [q% n (n)] k 

= [Q°k(N)] K . 

where Q° K (N) depends only of a neighborhood of <f>^ 1 (0). 

The orbit K ■ n° ^ N is an isotropic embedding since it is the 0-level of the 
moment map $at. Then to describe a K- invariant neighborhood of K ■ n° in N we 
can use the normal-form recipe of Marie, Guillemin and Sternberg. 

First we consider, following Weinstein (see [TTJEI]), the symplectic normal bun- 
dle 

(2.23) V := T(K ■ n°)^ u j T(K ■ n°) , 

where the orthogonal ( ± ' S1 ) is taken relatively to the symplectic 2-form. We have 

V = K x H V 

where the vector space V := T n o(K ■ u") 1 - 51 /T n o(K ■ n°) inherits a symplectic 

structure and an Hamiltonian action of the group H: we denote &h '■ V — ► f)* the 
corresponding moment map. 

Consider now the following symplectic manifold 

(2.24) N := V T*(K/H) = K x H ((t/t))* ® V 



The action of H on N is Hamiltonian and the moment map $^ : N — * 6* is given 
by the equation 

(2.25) = + k e K, £ £ (t/f))% v £ V . 

The Hamiltonian A"-manifold N is prcquantized by the line bundle Lfj\= A"x#C x . 



14 



PAUL-EMILE PARADAN 



The local normal form Theorem (see [10], [24] Proposition 2.5 ) tells us that 
there exists a l^-Hamiltonian isomorphism T : U\ — > U2 between a X-invariant 
neighborhood U\ of K ■ n° in N, and a if- invariant neighborhood of Jf/£f in 
N . This isomorphism T, when restricted to K ■ n°, corresponds to the natural 
isomorphism K ■ n° — > K/H. 

Thanks to T, we know that the fiber $^ 1 (0) C 1^ is reduced to {0}. This last 
point is equivalent to the fact that $h (and then is proper map (see [21]). We 
check easily that the set of critical points of ||3>jy|| 2 is reduced to $^(0) = K/H. 
Then, thank to the isomorphism T, we have that 



(2.26) 



Q°k(N) = Q° K (N) 



Let Ind^ : R-°°(H) 
relation (lnd H ((f>), E) = 



* R °°(K) be the induction map that is defined by the 
b,E\ H ) for any (f> G R-°°{H) and E E R(K). Note that 



[Ind H (0)] A ' = (Ind H (0),C) 



[*]*■ 



Since : V — > fj* is proper one can consider the quantization of the vector 
space V through the map <£ H : Q% H {V) G R-°°(H). 



Proposition 2.17. • We have 
(2.27) 



Q*/(N) =Ind K H (Q%»(V)®C x 



• The formal quantization Qf/ 1 (V) coincides, as a generalized H -module, to the 
H-module S(V*) of polynomial function on V. 

• The set [S(V*)] H of polynomials invariant by the connected component H° is 
reduced to the scalars. 



With the last Proposition we can finish the proof of Theorem 12.161 as follows. 
We have 



Q(M M ) 



K 



K 



Qk( n ) 
[S(V*)®C X \ 



H 



[C, 



Proof. The first point of Proposition 12.171 follows from the property of induction 
defined by Atiyah (see Section 3.4 in [19]). Let us explain the arguments. We work 
with the TJ-manifold y = (t/t))* ® V and the if-equivariant map j : y <—* N := 

K x H y,y^ [e,y]. 

We noticcQthat TN ~ K x H ($/f)®Ty), and that T^iV ~Kx H {T H y). Hence 
the map j induces an isomorphism j* '■ Kni^Hy) — * Kk(TrN). Theorem 4.1 of 



5 These identities come from the following K X i?-equivariant isomorphism of vector bundles 
over K x y-. T„(N) -> K x (t/fjeTy), (k, m; f t{t=0 (ke tx ) + v m ) ^ {k, m;pr m (X) + v m ). Here 

prt/fj : 6 — > 6/1) is the orthogonal projection. 
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Atiyah pQ tells us that the following diagram 



(2.28) K H (T H y) K K (T K N) 



Index, 



Index; 



R-°°(H) ^ R-°°(K) . 

is commutative. 

The tangent bundle TiV is cquivariantly diffcomorphic to 

Kx H [t/f) 8 (t/f))* ®TV]~Kx H [(t/f)) c e TV] 

where (t/f))c is the complexihcation of the real vector space t/f). We consider on 
TV the almost complex structure Jfj = (i, Jy) where i is the complex structure 
on (t/f))c and 3y is a compatible (constant) complex structure on the symplectic 
vector space V. Note that is compatible with the symplectic structure on a 
neighborhood U of the 0-section of the bundle N — + K/H. 
Let be the Kirwan vector field on iV: 

*jr([k;tM) = -t + i[e,* H (v)]®K V (y) G («/t»)c®V. 

Here is the Kirwan vector field relative to the Hamiltonian action of H on the 
symplectic vector space V. Note that vanishes exactly on the 0-section of the 
bundle N -► tf/ff. 

Let c K « be the symbol Thom(iV, J^) <E> Lf, pushed by the vector field k^. The 

generalized character Q^-™ (N) is either computed as the equivariant index of the 
symbols c K » or c K «|;y. 

Remark 2.18. The fact that is not compatible on the entire manifold N is 
not problematic, since Jjy is compatible in a neighborhood U of the set where 
vanishes. See the first point of Lemma \2.6l 

For X + in © w G T[ k .^ v ]N ~ (t/f))c © ^, the map 

(2.29) c K « (X + in © to) = c(x + £ + % - [£, 0c(i»- «v(w)) 

acts on the vector space Ac(t/f))c ® Aj^F <8> C x . 

Let Bott(t/f)) be the Bott morphism of the vector space t/f). It is an elliptic 
morphism defined by 

Bott(t/t))\ 6 (r}) = c(£ + if]) acting on A c (t/J))c, 

for rj G T 5 (t/f)). Let c Kv " be the symbol Thom(V, Jy) pushed by the vector field 

Lemma 2.19. FFe /iai/e 

c K « = j» (Bott(t/f)) © c Kv <g> C x ) . 
Proof. We work with the symbol 

* T |(€,«) fo) = c(£ + «77 - »T K, *if («)]) 
acting on A c (t/f))c. Note that Bott(t/f)) = tj°. From (|2~29|) . we see that c K « = 
j* (V c Kv (g) C X J . It is now easy to check that a T c Kv g) C x , T G [0, 1] is an 
homotopy of transversally elliptic symbols on t/f) x V. □ 
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The commutative diagram (|2.28|) and the last Lemma gives 

Q**(iV) = Indcx^(c K «) 

= Ind* (lndcx^ xy (Bott(t/fj) © c Kl ') $ C x ) 

= Ind* (lndcx^(Bott(Vf))) <g> Index£(c rev ) <g> C x ) 

= In<C(Q% H (V)®C x ). 

We have used here that the equivariant index of Bott(t/f)) is equal to 1 (e.g. the 
trivial representation) . 

Let us proved now the second point of Proposition ^. 171 The Kirwan vector field 
k v satisfies the simple rule: 

(2.30) ( K v (v),J v v) = -n(K V (v),v) = ^\\<S> H (v)\\ 2 , veV. 

It shows in particular that k v {v) = <S> H (v) = 0. Since the moment map 
&h : V — > f)* is quadratic, the fact that is proper is equivalent to the fact that 

**(o) = o. 

We consider on V the family of symbol a s : 

cr s \ v (w) = c(^w — sk v (v) — (1 — s)Jyv S J 

viewed as a map from A c vcn V^ to A c dt V. Thanks to (pOO]) . one sees that ct s is 
a family of if-transversally elliptic symbol on V. Hence a 1 = c Kv and er° = 
c(u> — Jyv) defines the same class in the group Kk(TkV). The symbol er° was first 
studied by Atiyah [T] when dime V = 1 . The author considered the general case in 
HSI. We have 

Indexed ) = in R-°°(K). 

The last point of Proposition 12.171 is a consequence of the properness of the 
moment map &h (see Section 5 of [2"Tj). 

□ 



□ 



Example 2.20 ([21]). We consider the action of the unitary group U n on C n . 
The symplectic form on C" is defined by Cl(v,w) = ^.'Ylik v k u ^k — VkWk- Let us 
identify the Lie algebra u„ with its dual through the trace map. The moment map 
$ : C™ — > u„ is defined by &(v) = (g) v* where v ® v* : C" — > C" is i/ie linear 
map w i— > (^2 k VkWk)v. One checks easily that the pull-back by 4> o/ a \J n -orbit in 
u n is either empty or a U n - orbit in C™. We knows also that the stabiliser subgroup 
of a non-zero vector of C" is connected since it is diffeomorphic to U n _i. Finally 
we have 



(2.31) Q((C%) 



1 if /i € U„ belongs to the image of $ 

if fx € U„ does not belongs to the image of $. 



Then one checks that Su^°(C") coincides in R °° (U n ) ww'tt i/ie algebra S((C 1 )*) 
of polynomial function on C" . 
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Example 2.21 (|23|). We consider the Lie group SL2(M) and its compact torus 
of dimension 1 denoted by T. The Lie algebra sbQR) is identified with its dual 
through the trace map, and the Lie algebra t is naturally identified with s[2(K) T . 
For I G Z\ {0}, we consider the character \i of T defined by 



cos a — sin t 
Xl 1 sin 9 cos 9 I = 6 



Its differential \dxi G t* correspond (through the trace map) to the matrix 

Xl -{ -1/2 ) ■ 

Let Oi be the coadjoint orbit of the group SL2 (M) trough the matrix Xi . It is a 
Hamiltonian SL2(M) -manifold prequantized by the SL2 (M.)-equivariant line bundle 
Li ~ SL2(M) Xj- Ci, where C/ is the T -module associated to the character xi- We 
look at the Hamiltonian action of T on Oi. Let $^ ■ Of - * t* be the corresponding 
moment map. One checks that the moment map $r is proper and that its image 
is equal to the half-line {aXi, a > 1} C t*. 

We check that for each £ G {aXi,a > 1} the fiber $~r (£) is equal to a T -orbit 
in Oi. For k G Z, let us denote (Oi) k the symplectic reduction of Oi at the level 
Xk- We knows that (Oi) k — if k (£ {al,a > 1}, and that (Oi) k is a point if 
k G {al, a > 1}. 

In order to compute Q({Oi) k ) we look at the stabilizer subgroup T m := {t G 
T 1 1 • m = m} for each point m G Oi . One sees that T m = T if m = Xi and T m is 
equal to the center {±Id} o/SL^R), when m 7^ Xi. 

Theorem \2.16\ gives in this setting that, for k G {al, a > 1}, 



(2-32) C((Oi) fc ) 



1 if I — k is even 
if I — k is odd. 



Hence the formal geometric quantization of the proper T -manifold Oi is 

(2.33) e?-(0 = (?'g^ 

[C; -Ep^oC-ap ifl<0. 

_ffere we recognizes that Q^ 00 (Cz) coincides with the restriction of the holomorphic 
(resp. anti-holomorphic) discrete series representation Qi to the group T when 
I > (resp. I < 0). 

2.5. Wonderful compactifications and symplectic cuts. Another equivalent 
definition of the quantization Q~°° uses a generalisation of the technique of sym- 
plectic cutting (originally due to Lerman |13j ) that was introduced in [21] and was 
motivated by the wonderful compactifications of De Concini and Proccsi. Let us 
recall the method. 

We recall that T is a maximal torus in the compact connected Lie group K, and 
W is the Weyl group. We define a K-adapted polytope in t* to be a M^-invariant 
Delzant polytope P in t* whose vertices are regular elements of the weight lattice 
A*. If {Ai, . . . , Xn} are the dominant weights lying in the union of all the closed 
one-dimensional faces of P, then there is a G x G-equivariant embedding of G = Kc 
into 

N 



1=1 
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associating to g G G its representation on V\ t . The closure Xp of the image 

of G in this projective space is smooth and is equipped with a if x if that we 
denote: 

(fci, £2) • x = &2 • x ■ k^ 1 . 

Let fi^p be the symplectic 2-form on Xp which given by the Kahler structure. We 
recall briefly the different properties of {Xp, Slx P ) : all the details can be found in 

M- 

(1) Xp is equipped with an Hamiltonian action of if x if . Let $ = (<!>;, $ r ) : 
M — > 6* x t* be the corresponding moment map. 

(2) The image of $ := ($/, $ r ) is equal to {(fc-f, -k'-£) | £ G P and fc, k' G if}. 

(3) The Hamiltonian manifold (Ap, if x if) has no multiplicities: the pull-back 
by $ of a A' x if -orbit in the image is a if x if -orbit in Xp. 

Let Up := if ■ P° where P° is the interior of P. We define 

X° P -if?- 1 (Up) 

which is an invariant, open and dense subset of Xp. We have the following impor- 
tant property concerning Xp. 

(4) There exists an equivariant diffeomorphism T : if X Up — ► A"p such that 
T*(*,)(*,0 =k-t and T*($ r )(fc,0 = 

(5) This diffeomorphism T is a quasi-symplectomorphism in the sense that 
there is a homotopy of symplectic forms taking the symplectic form on the 
open subset K x Up of the cotangent bundle T*K to the pullback of the 
symplectic form ttx P 

on Xp . 

(6) The symplectic manifold (Xp,Qx P ) is prcquantized by the restriction of 
the hyperplanc line bundle 0(1) -> P(®^ =1 F A * ® VxJ to X P : let us denoted 
Lp the corresponding K x if-equivariant line bundle. 

(7) The pull-back of the line bundle Lp by the map T : K x Up Ap is 
trivial. 

Let (M, f2jvf , $m) be a proper Hamiltonian A'-manifold. We also consider the 
Hamiltonian K x if-manifold Xp associated to a if -adapted polytope P. We con- 
sider now the product M x Xp with the following K x K action: 

• the action k -\ (m, x) = (k ■ m, x ■ fc _1 ) : the corresponding moment map is 
$i(to,x) = $ M (m) + $ r (x), 

• the action k -2 (m, x) = (to, k ■ x) : the corresponding moment map is 

$ 2 (to,x) = 

Definition 2.22. We denote Alp the symplectic reduction at of M x Ap /or i/ie 
actton -i : Ai P := ($i)- 1 (0)/(if, -i). 

Then Mp inherits a Hamiltonian if-action with moment map $m p : Mp ~ * 6* 
whose image is <£>(Ai) D K ■ P. 

One checks that Mp contains an open and dense subset of smooth points which 
quasi-symplectomorphic to the open subset (&m)~ 1 (Up). If the polytope P is fixed, 
we can work with the dilated polytopes nP for n > 1. We have then the family of 
compact, perhaps singular, if -hamiltonian manifolds M n p, n > 1: in Section 12.31 
we have explained how was defined their geometric quantization Qk (M n p) G R(K). 

We have a convenient definition for Q~°°. 
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Proposition 2.23 ([21]). We have the following equality in R °°{K): 

(2.34) Qx°°(M) = lim Q K (M nP ). 

n — >oo 

3. Proof of Theorem 11.41 

The main result of this section is 

Theorem 3.1. Let rp := inf^ggp The generalized character 

Q|(M) - Qk(M p ) e R-°°(K) 

is supported outside the ball B rp . 

Then, for the dilated polytope nP, n > 1, the character Q%(M) — Q K {M n p) is 
supported outside the ball B nrp . Taking the limit when n goes to infinity gives 

(3.35) Qfc(M) = lim Q K (M nP ). 

n — >oo 

Finally, the identity of Theorem 1 1.41 

Q|(M) = QJTM, 

is a direct consequence of (|2.34[) and (|3.35p . 

Recall that 0(r) G R~°°(K) denoted any generalized character supported out- 
side the ball B r . 

Theorem 13.11 follows from the comparison of three differents geometrical situa- 
tion. All of them concern Hamiltonian actions of K\ x where K\ and K 2 are 
two copies of K. 

First setting. We work with the Hamiltonian K\ x ^-manifold M x Xp\ 
here K\ acts both on M and on Xp. Since the moment map $1 (relative to the 
ifi-action) is proper we may "quantize" M x X P via the map ||$i|| 2 : let 

Q%\ xK2 (MxX P )eR-°°(K 1 xK 2 ) 

be the corresponding generalized character. Recall that Qk 2 (Mp) is equal to 

Second setting. We consider the same setting than before : the Hamiltonian 
action of K\ x K2 on M x Xp. But we "quantize" M x Xp through the global 
moment map $ = ($1, $2)- Here we have some liberty in the choice of the scalar 
product on 6* x 63- 11 ||£|| 2 is an invariant Euclidean norm on t* , we take on i\ x 63 
the Euclidean norm 

(3.36) ll(a,6)llM£ill 2 + HMI 2 

depending on a parameter p > 0. Let us consider the quantization of M x Xp via 
the map \\$\\ 2 p : 

Qk?*k 2 (M x X P ) G J?-°°(iTi x K 2 ). 

Third setting. We consider the cotangent bundle T*K with the Hamiltonian 
action of K\ x K^: K\ acts by right translations, and K2 by left translations. We 
consider the Hamiltonian action of K\ x K2 on M x T*K : here K\ acts both on 
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M and on T*K. Let $ = (<E»i, $ 2 ) be the global moment map on M x T*K. Since 
the moment map $ is proper we can "quantize" M xT*K via the map 1 1 <E> 1 1 ^ : let 

Qk P *K 2 (M x T*K) G i? _oc (A'i x ET 2 ) 
be the corresponding generalized character. 

Theorem 13. II is a consequence of the following propositions. 
First we compare Q% 2 (M) with the ifi-invariant part of Q^ xK ^{M x T*iT). 
Proposition 3.2. For any p <g]0, 1], we have 

(3.37) [ Q ^ xK2 (MxT*iOp = Q| 2 (Af) in R~°°{K 2 ). 

Then we compare the Xi-invariant part of the generalized characters 
2^W M x T*K) and Q% xK ,(M X 

Proposition 3.3. For any p e]0, 1], we have the following relation in R~°°(K 2 ) 



(3.38) [QY^k 2 {M x X P )\ ■ [Q^ xK2 (MxT*K)\ = 0{r P ) 

Finally we compare the Xi-invariant part of the generalized characters 
2*;W M x *p) and Q%\,k 2 (M x X P ). 



Proposition 3.4. There exists e > such that 

'-K 1 xK 2 



(3.39) Qk 2 (A/ p )- fe xif ,(Mx ^p)l Al =0((e/ P ) 1 / 2 ) in ir°°(ET 2 ) 



if p > is small enough. 

If we sum the relations (|3.37|) , (|3.38[) and (|3.39[) we get 

S| 2 (M) = Qk 2 (M p ) + 0(r P ) + 0((e/p) 1/2 ) 
if p is small enough. So Theorem 13.11 follows by taking (e/p) 1 / 2 > rp. 

3.1. Proof of Proposition 13.21 The cotangent bundle T*K is identified with 
K x 6*. The data is then (see Section [5Tj) : 

• the Liouville 1-form A = ■ <g> Ej. Here (Ej) is a basis of t with dual basis 
(Ej), and luj is the left invariant 1-form on K defined by LUj(-^ae tx \ Q ) — (E* 7 X). 

• the symplcctic form Q := —dX, 

• the action of K\ x K 2 on K x 6* is (k\,k 2 ) ■ (a, £) = (fc 2 a/cf 1 , fci ■ £), 

• the moment map relative to the E^-action is <£v(a,£) = — £, 

• the moment map relative to the ET 2 -action is $;(a,£) = a - £. 

We work now with the Hamiltonian action of K\ x K 2 on MxT'Jf given by 
(ki,k 2 ) ■ (m,a,£) = (ki ■ m,k 2 ak^ ,ki ■ £). 
The corresponding moment map is $ = ($i,$ 2 ): <&i(m, a, £) = $a/(to) — £ and 

Let ci be a symbol Thom(M, J\)<E) L attached to the prequantized Hamiltonian 
ETi-manifold (M, 0). The cotangent bundle T*ET is prequantized by the trivial 
line bundle: let c 2 be the symbol Thom(T*ET, J 2 ) attached to the prequantized 
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Hamiltonian K\ x ^-manifold T*K . The product c = ci C2 corresponds to the 
symbol Thom(7V, J) <g> L on N = M X T*K. 

Let k p be the Kirwan vector field associated to the map \\$\\ 2 p : AI x T*_K~ — > M. 
We check that ||$|| 2 (™, A:, = ||$ A /(m) - £|| 2 + pj|£|| 2 : and 

K p (m,k,£) = ( faW-Q'm ; - (1 + p)| ; -[$ M (m),^ 

Here T (m)fcif) (M x T*^) ~ T m M xtxE. Wc have 



2 ) = x^p = u; 





Cr(||$||P = {« P = 0} 



|J Ki x K 2 

/3GB 



M'»n^ 1 C9)x{i}x{^ I } 



where B parametrizes Cr(||$M|| 2 )- Hence one checks that the critical values of ||$|jp 
are^||/?|| 2 ,/3G£. 

Let c Kp be the symbol c pushed by the vector field K p : we have 

c K " (v;X;Y) = d(v - Ki) Q c 2 (X - kj, p ;Y - k m ) 

for (v;X;Y) G T (mM (M x T*X) ~ T m M xEx!. 

For a real i? > we define the open invariant subsets of M x T*K 

U R := {\M 2 P <R} 

Vr := {||$jw|| 2 < -R} x T*K. 

By definition the generalized index Qjc^xK 2 (^ x T*if) is defined as the limit of 
the equivariant index 

Q^k 2 (Ur) :=Indcx^ x ^(c K "kJ, 

when R goes to infinity (and stays outside the critical values of ||$||p). 

In the other hand, when R' is a regular value of ||$a/|| 2 , we see that the symbol 
c p \v R , is K\ x X 2 -transversally elliptic. Let 

(3.40) Indcx^^c^,) 

be its equivariant index. Notice that the index map is well-defined on Vr = 
{\\®m\\ 2 < R} x T*K since T*K can be seen as a open subset of a compact 
manifold. 

It is easy to check that for any R > there exists R' > R such that Ur C V' r . 
It implies that Qk? xK A M x T * K ) is also defined as the limit of (|3T4"0|l when R' 
goes to infinity. 

We look now to the deformation k p (s) = (/if; k s e ; skm), s £ [0, 1] where 

K/(m,£) = ( $ mM- s£)-m and Kj >p (m,f) = s$ M (m) - (1 + sp)f. 
Let c^ 8 ' be the symbol c pushed by the vector field k p (s). 

Lemma 3.5. Lei i?' be a regular value of ||<!>m|| 2 - 

• The familly c Kp ^ \y , , s £ [0, 1] defines an homotopy of K\ x K2-transversally 
elliptic symbols on Vri . 

• The Kx-invariant part o/Index^f^^c^^lvjjJ is e 1 ual to Q% i M <R')- 
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Proof. The first point follows from the fact that Cha^c*?^ \ Vr , ) H T Kl xk 2 (Vr/), 
which is equal to 

(m,fc, —^—^(m)), fc e X and m e Cr(||$ M || 2 ) n {||$m|| 2 < #} 
1 + sp 

stays in a compact set when s € [0,1]. 

The symbol c re "(°)|y , is equal to the product of the symbol c*\m<r', which is 
ifi-transversally elliptic, with the symbol 

c%(X;Y) = c 2 (X + £;Y) 

which is a A"2-transversally elliptic on T*A. A basic computation done in section 
15.1.21 gives that 

Index^ K2 (c£) = L 2 (A) 

IJ.GK 

in R~°°(Ki x A 2 ). Finally the "multiplicative property" (see Theorem 12. 1[) gives 
Index^ 1 ,* K2 (c K » W\ Vr/ ) = Index^ R , (c? \m<r>)® Index* 1 (c 2 «) 

= E Gx 1 (^<H')®« i r®^ fa 

Taking the Ai-invariant completes the proof of the second point. □ 

Finally we have proved that the generalized character [Index*^* * 2 (c Kp \y Rl )] Kl 
is equal to Q*, (M<#'). Taking the limit R' — > oo gives 



Q%xk,(MxT*K) = km Index™ (c^,) 



A', 



A, 



ii'-»oo 
it/— >oo 

3.2. Proof of Proposition 13.31 We work here with the Hamiltonian action of 
K\ x K2 on M x <Yp. The action is (fci, k 2 ) ■ (m, x) = (k ■ m, k 2 ■ x ■ fcf 1 ) and the 
corresponding moment map is $ = ($i,$ 2 ) with $i(m,x) = <&a/(to) + § r {x) and 
$ 2 (m,a;) = Let |j(£i,6)llp = II6II 2 + PII6II 2 be the Euclidean norm 8J x fi* 

attached to p > 0. 

Let us consider the quantization of M x Xp via the map H^H 2 ,: 

Q^K 2 (M^X P )eR-°°(K 1 x A 2 ) 

The critical set Cr(|j$|| 2 ) admits the decomposition 

(3.41) Cr(||$||2)= (J A' lX A 2 -C 7 

where (m, x) € C 7 if and only if 7 = (71, 72) with 

'$ M (m) + $ r (x) = 71 
$z(x) = 72 
71 ■ m = 

k 7l - r X + p7 2 X = 0. 



(3.42) 
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We have 
(3.43) 



e&W M x *p) = E Qk:><k 2 (m x x P ) 

ieB P 



where the generalized character Q^f x jf 2 (-^ x <^p) is computed as an index of a 
transversally elliptic symbol in a neighborhood of 

K\ x K 2 ■ C 7 c M x ^\K 2 • 72 ). 

Thanks to Theorem 12.91 we know that the support of the generalized character 

QkixK 2 ( M x Xp) is contained in {(a, b) £ K± X K2 I ||a|| 2 + p||&|| 2 > blip}- Hence 

support ([QY lXK2 (M x X P ))] Kl ) c {b e K 2 \ P \\b\\ 2 > \\ 7 f p ] 

Let rp = inf^ggp ||£||. We know then that 



Q 



K ± xK 2 



(M x X P ) 



E [QK P 1 xK 2 (MxX P )] Kl +0(r P ) 



\h\\ 2 p <pr 2 f 



Let Rp < prp be a regular value of ||$|jp : M x Ap — ► R such that for all 7 G 23 p 



we have H7H;; < prp 



\-y\\i < i? P . Then 



(3.44) 



Ai 



G^W^x**) - Qk?xk 2 ((M * Xp)<R p ) +0(r P ) 



For the gcner alized index Q^xK 2 ( M x t *^) 

we have also a decomposition 
G&WM x T*#) = 2 C^ XX8 (M x T*JO 

where S p parametrizes the critical set of ||$||p : A/ x T*K — ► M. Like before we get 



(3-45) Q^ Kj (MxW 



A"i 



Q*k?xk 2 ((MxT*K) <r , p )\ +0(r P ). 

Here i? P < prp is a regular value of : M x T*K — > R such that for all 7 G B p 
we have H7H 2 , < prp ||7||2 < i? p . 

Lemma 3.6. We have 

(3-46) Q*;W( M x <%»<«p) = C^f xJfa ((M x T*K) <R , p ). 

Proof. The Lemma will follow from Proposition 12.61 We take here V' = M x Xp, 
V = M X K X Up C M X T*K and the equivariant diffeomorphism * : V — ► V is 
equal to Id x T where T was introduced in Section [2~5l Note that iff satisfies points 
(1) - (3) of Proposition EH 

Note that ||5>(m, x)\\ 2 p < prp implies that ||$z(x)|| < r P and then x £ Xp. Hence 
the open subset U' := (M x Xp) < p p is contained in V' = M x Xp. In the same 
way the open subset U := [M x T*K) <R ' is contained in V. We have ^f(U) = U' 



if R f 



R'p 



We have proved that (|3.46|) is a consequence of Proposition 12.61 
Finally, if we take the difference between (|3.44|) and (|3.45j) , we get 



□ 



Q*;W M x Xp)] Kl [QZxK 2 (M x T*K) 
which is the relation of Proposition 13.31 



0(r P ) 
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3.3. Proof of Proposition 13.41 Here we want to compare the if i-invariant part 
of the characters Q^ xKz (M x X P ) and Q%\ xK2 (M x X P ). 
We know after Theorem 12. 141 that 

Qk 2 (Mp) = [Q%\ xK2 (Mx X P )] Kl 

= 'Q%\*kM)] K1 

when e > is any regular value of ||$i|| 2 , and U € := {||$i|| 2 < e} C M x X P . 
In this section we fix once for all e > small enough so that 

(3.47) Crdl^H 2 ) n {H^SI 2 < e} = {$ x = 0}. 

Let Ci be the symbol Thom(M, Ji) ® L attached to the prequantized Hamilton- 
ian ifi-manifold (M, f2). Let C3 be the symbol Thom^p, J3) ® Lp attached to 
the prequantized Hamiltonian A"i x ^-manifold Ap. The product c = ci C3 
corresponds to the symbol Thom(iV, J) L on N = M x X P . 

Let /«o and k p be the Kirwan vector fields associated to the functions ||$i|| 2 and 
||$|| 2 on M x X P : 

Ko(m, x) = ^ $i(to, x) ■ m ; x) > x^ , K p (m, x) = K°(m, x)+p (0, $;(x) a;). 

KI Ke Km 

Let c K ' > be the symbol c pushed by the vector field k p : we have 
c Kp (v; n) = a(v - ki) c 3 (r/ - Kj — pK M ) 

for (wjij) e T( m ,x){ M x 

The character S* 1 ^^^) is given by the index of the ifi-transversally elliptic 
symbol c K ° | jj e ■ The character Q^ x K2 (M x X P ) is given by the index of the K\ x K2- 
transversally elliptic symbol c Kp . 

Lemma 3.7. • There exists p(e) > such that 

Cr(||$|pn{Pill 2 <4c{||$i|| 2 <|} 
for any < p < p(e). 

Proof. With the help of Riemannian metrics on M and Ap we define 
a(e) := inf II k (m.x) II 

e/2<||*i(m,a:)||<£ 

b := sup ||$ ; (x) - t x\\. 

xEXp 

We have a(e) > thanks to (|3.47p , and b < 00 since Ap is compact. It is now easy 
to check that {n p = 0} n {e/2 < ||$i|| 2 < e} = if < p < 

□ 

The symbols c Kp \u e , p € [0,p(c)] arc K\ x if 2 -transversally elliptic, and they 
define the same class in KK 1 xK 2 { r ^K 1 xK 2 U € ). Hence Qk 2 (M p ) can be computed 
as the if i-invariant part of 

for pe[0, /9(e)]. 
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A component K\ x K 2 ■ C 7 of Cr(||$|| 2 ) is contained in U e if and only ||7i|| < e : 
hence the decomposition (|3.43p for the character Q^'f x jf 2 (-^ x Xp) gives 

Qk^k 2 (MxX p ) = Q^ xK2 (U £ )+ Qk P iX k 2 (M x X P ). 

Il7l|| 2 >e 

where 

QkUM)= E QktxK^M x X P ). 

Il7i|| 2 <e 

Taking the Ai-invariant gives 
(3-48) [Q% xKa (MxX P )] K i=Q Ka (M P ) + E IQk%k 2 (Mx X P )] K K 

Il7i|| 2 >« 

In general we know that the support of the generalized character 
IQkixk 2 ( M x X p))\ Ki is included in {b e K 2 \ p\\b\\ 2 > || 7l || 2 + HI72II 2 }. When 
||7i|| 2 > e we have then that the support of [Q^f x ^ 2 (Af x Xp))] Kl is contained in 

{beK 2 \ p\\b\\ 2 >e}. 

Finally (|3.48p imposes that 

[Q&W M x Xp)] k ^ = Qk 2 (M p ) + 0{{e/p) 1 ' 2 ). 
when < p < p(e) , which is the precise content of Proposition 13.41 

4. Other properties of Q $ 

Let (M,u, $) be a proper Hamitlonian A"-manifold which is prequantized by a 
line bundle L. The character Q* (M) is computed by means of a scalar product on 
6*. The fact that Q%(M) = Q^°°(M) gives the following- 
Proposition 4.1. The character Q%{M) does not depend of the choice of a scalar 
product on t* 

In this section we work in the setting where K = K\ x K 2 . Let $1 be the moment 
map relative to the AVaction. 

4.1. $1 is proper. In this subsection we suppose that the moment map $1 relative 
to the Ai-action is proper. We fix an invariant Euclidean norm |j • || 2 on t in such 
a way that 61 = t 2 . 

Let us "quantize" (M, f2) via the invariant proper function ||$i|| 2 : let 

Qk\xk 2 (M) e R~°°(Ki x K 2 ) 
be the corresponding generalized character. 
Theorem 4.2. We have 

(4-49) Q% 1XK2 (M) = Q%\ XK2 (M) in ir°°(Ax X K 2 ). 
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Proof. On t = t± © t 2 we may consider the family of invariant Euclidean norms: 
WXi (£ X 2 \\ 2 p = \\XiW 2 + p\\X 2 \\ 2 forXj et 3 -. Let 

be the quantization of M computed via the map \\^>\\ 2 = jl^iH 2 + p|| ( E > 2|| 2 - By 
definition, Qjc iX k 2 {-^) ^ s equal to Q^ x k 2 (M), and we know after Proposition ^. II 
that Qk 1 xk 2 (M) coincides with the generalized character Q^ x ^ (M) G R~°°(K) 
for any p > 0. 

Let us prove that prove that Q^xK (M) = Q'k 1 xk 2 (M)- We denote 0(r) E 
R~ 00 (K 1 x K 2 ) any generalized character supported outside the ball 

tfet; xt; 1 ||6ll 2 + H6ll 2 <r 2 }. 

And we denote Oi(r) € R~°°(Ki x K2) any generalized character supported outside 
the 

Uettxtsl |Ki||<r}. 

Let i?i > be a regular value of ||$i|| 2 : the open subset {||$i|| 2 < Ri} is 
denoted M < r 1 . We know that 

Q%\xk 2 (M) = Q%\xk 2 (M<R 1 ) + Oi(VR~i)- 
Like in the Lemma 13.71 we know that 
(4.50) Cr(||$|| 2 )n{||a> 1 || 2 = i? 1 }=0. 

for p > small enough. The identity (|4.50[) first implies that 

Q*k?xk 2 (M) = £ Qk'xk 2 (M)+ £ QY^k 2 (M) 

Il7i|| 2 <-Ri Il7i|| 2 >-Ri 



= Qk^kAMkrJ + OWRx). 

In the second equality we have used that Qjc^xK 2 (^) = O(v^Ri) when ||7i|| 2 > i?i 
since the ball {(6,6) e t| X t$ | ||6l| 2 + II6II 2 < Ri} is contained in 

{(6,6) et^ I 11(6,6)11? < 11(71,72)11?}. 

The identity (|4.50p shows also that the symbol c k i'\m<Ri are homotopic for p > 
small enough. Hence 

Qk P xK 2 (M<R 1 ) = Q%\xK 2 (M <Rl ) 

We get finally that Q%? xKa (M) - Q^^fM) = 0(>/Ei) + Oi(VHI) for any 
regular value i?i of ||$i|| 2 . We have proved that Ql^^M) - Q%\ X K 2 ( M ) = 
0. □ 



Let us explain how Theorem l4.2l contains the identity that we called "quantization 
commutes with reduction in the singular setting" in [21] . By definition the K\- 
invariant part of the right hand side of (|4.49|) is equal to the geometric quantization 
of the (possibly singular) compact Hamiltonian ^-manifold 

M//.K, ^^(Oj/tfi. 
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Using now the fact that the left hand side of (|4.49p is equal to Qjc^xK 2 (M), we see 
that the multiplicity of V^ 2 in Qk 2 (M// q Ki) is equal to the geometric quantization 
of the (possibly singular) compact manifold 

M xK^L/f^K! x K 2 . 

4.2. The symplectic reduction M// Ki is smooth. Let (M, Si) be an Hamil- 
tonian K\ x ^-manifold with a proper moment map <f> = (<&i, $2)- In this section 
we suppose that is a regular value of $1 and that K\ acts freely on $ 1 " 1 (0). We 
work then with the (smooth) Hamiltonian ^-manifold 

We still denote by $2 : N — > t 2 the moment map relative to the ^-action: note 
that this map is proper. Hence we can quantize the ^-action on N via the map 
$2. Let Q% 2 2 (N) e R~°°(K 2 ) be the corresponding character. 

Proposition 4.3. We have 

(4-51) [Q% ixK2 (M)] Kl = Q%l(N) in R~°° (K 2 ). 

Proof. When $1 is proper, the manifold N is compact. Then the right hand side of 
(|4.51|) is equal to Qk 2 (N), and we know from Thcorem l4.2l that the left hand side of 
(|43T|) is equal to [Q%\ xK ,(M)] Kl . In this case (|43T|) becomes [Q%\ xKa (M)] Kl = 
Qk 2 (M // Q Ki) which is the content of Theorem 12. 131 

Let us consider the general case where $1 is not proper. Thanks to Theorem 
[Clone knows that the multiplicities of V* 2 in [Q% lxK2 (M)] Kl and Q%\(N) 
are respectively equal to the quantization of the (possibly singular) symplectic 
reductions 

M» := M x K 2 ■ fJL// {m K x x K 2 . 

and 

M'^:=N x K2~jI// K 2 , with N = M // Q K\. 

Note that M.^ and M.'^ coincide as symplectic reduced space. Let us prove that 
their geometric quantization are identical also. The proof will be done for \i = 0: 
the other case follows from the shifting trick. 

Let c be the K\ x i^-equivariant symbol Thom(M, J) ® Lm- Let k be the 
Kirwan vector field attached to the moment map $ = (&i,& 2 ). Let c K be the 
symbol c pushed by k. Let us denote M <e the open subset {H'&ll 2 < e}. For e > 
small enough, the symbol c k \m <€ is K\ x i^-transversally elliptic, and Q(A4o) is 
the K\ x ^-invariant part of Index^*^ 2 (c k |a/ <5 ). 

Let C2 be the -KVequivariant symbol Thom(iV, J) ® Lm- Let k 2 be the Kirwan 
vector field attached to the moment map $2- Let c^ 2 be the symbol C2 pushed by 
k 2 . Let us denote iV <£ the open subset {|| ( E > 2|| 2 < e}- For e > small enough, the 
symbol C2 2 |jy <e is ^2-transversally elliptic, and Q(Ai' ) is the ^-invariant part of 
Indcx^ 2 e (c« 2 U <s ). 

Our proof follows from the comparison of the classes 

[c K | M<e ] G K KlxK2 (T KlxK2 M <e ) 

and 

[<%>\ N<m ]eK Ka (T Ka N <e ) 
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A neighborhood of the smooth submanifold Z := <I> 1 " 1 (0) in M is diffcomorphic to 
a neighborhood of the 0-section of the bundle Z x 6* — * Z. Let Z <e = Z n M <e so 
that N <e = Z <t jK\. Hence [c k |a/ <5 ] can be seen naturally a class in the K-group 
'K KlxK2 (T KlxK2 (Z <e x I*)). 

Following Atiyah [1] [Theorem 4.3], the inclusion map j : Z <e Z <e x t* induces 
the Thorn isomorphism 



j! : Kif lX if 2 (Tif lX x 2 ^< e 
with the commutative diagram 



(4.52) 



K 



(T Xl 



Xf(/<f 



Index 



KK lX K 2 (Tx lX K 2 (Z <e x t*)) 



Index**** 2 , 




i?-°°(A'i x A 2 



Let 7Tx : Z <t — > A <£ be the quotient relative to the free action of K\. The 
corresponding isomorphism 



ni:K K2 (T K2 N <e ) 
satisfies the following rule : 



K 



K 1 xK 2 {^K 1 xK 2 



,.Z<e) 



Index 



K 1 xK 2 



«0) 



(4.53) 

for &ny 9 eK K2 {T K2 N <t ). 
Lemma 4.4 ([15]). We have 

3\ 07r i( [ c 2 2 k< 
m K KlxK2 (T KlxK2 (Z <e x 6f)). 



lndex£<*(0) 



K\m <c 



Proof. This Lemma is proven in \W\ [Section 6.2] when the group Ki is trivial. It 
is easy to check that the proof extends naturally to our setting. □ 

If one uses Lemma T4.4I together with (|4.52[) and (|4.53p . we get that 



Q(M ) 



Indexf 1 *^ 2 



K x xK 2 



A 2 



Index* 2 ^[^J =Q(M' ) 



□ 



5. Example: the cotangent bundle of an orbit 

5.1. The formal quantization of T* A . Let A be a compact connected Lie group 
equipped with the action of two copies of A: (fci, A^) ■ a = k^ak^ 1 ■ Then we have 
a Hamiltonian action of K\ x A2 on the cotangent bundle T*A. In this section, 
we check that each formal geometric quantization of T*A, Qk^xK (T*A) and 
Q* ixX2 (T*A), are both equal to the Ai x AVmodule L 2 (K). 
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The tangent bundle Tif is identified with A' x t through the right translations: 
to (a, X) e K x 6 we associate ^ae tx |o- The action of ifi x if 2 on the cotangent 
bundle H*K ~if xf is then 

(fci, fc 2 ) • (a, £) = (k 2 ak^ 1 ,k 1 ■ £). 

The symplectic form on T*A' is 17 := — dA, where A is the Liouville 1-form. Let 
us compute these two form in coordinates. The tangent bundle of T*if ~ K x fi* 
is identified with T*if xf xf: for each (a, £) G T*if, we have a two form fi( ,£) 
on t x F. A direct computation gives 

n (0iC) (x,y) = y]>, n M (v,rf) = o, ii M {x, v ) = ( v ,x) 

for X, y G t and 77,?/ G 6*. So ^(a,£) = ^0 + tt£ where fio is the canonical 
(constant) symplectic form on t x 6* and tt^ is the closed two form on 6 defined by 
*t{X,Y) = {Z, [X,Y]). 

If we identify 6 ~ 6* through an invariant Euclidean norm, the symplectic struc- 
ture on T( 0i f)(T*-K') — 6 X 6* is given by a skew-symmetric matrix 

ad(f) /„ 

-In 

We will work with the following compatible almost complex structure on the tangent 
bundle of T*if : J$ = -A^(-A^ 1/2 . When £ = 0, the complex structure J on 
i x 6* is defined by the matrix 



Jo 



-I n 

In 



Hence the complex if -module (6 x 6*, Jo) is naturally identified with the complcx- 
ification 6c of £. 

One checks easily that the moment map relative to the ifi x if 2 -action is the 
proper map $ : T*if —> x 6 2 defined by $(a, £) = (— £, a ■ £). 

Here the symplectic manifold T*if is prequantized by the trivial line bundle. 



5.1.1. Computation of Q K ^° xK (T*if). Let 0\ x 2 be a coadjoint orbit of ifi x if 2 
in 6* x ?2- One checks that 



(5.54) X C 2 ) 



if Oi 54 -Oa 

a ifi x if 2 - orbit if d = -0 2 



We knows that the stabiliser subgroup ifj of an element £ e 6* is connected. 
Then the stabilizer subgroup (ifi x if 2 )( 0) £) = {(fci, afcia -1 ), fci G iff} is also 
connected. 

Let (T* K)^ x) be the symplectic reduction of T*if at the level A) G if 2 . 
For any fi G if, we define /1* G if by the relation —if • /i = if • /i*: note that 
V/fJ ~ (V/f )*. If one uses Theorem 12. 16[ one has 



(5-55) C((T*JQ(m.a)) 



if A ^ /i* 

1 if A = /i* 
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Finally 

5.1.2. Computation of Q% iX k 2 (T*K). The Kirwan vector field on T*K is 

K (a,£) = -2£et c . 

Let c K be the symbol Thom(T*A, J) pushed by the vector field \n. At each 
(a,£) G T*A, the map cf Q?) (A © 77) from A^' e,l (e x r) to A^ rf (« x t*) is equal 
to the Clifford map c(X + £ © rj). Note that c K is a A2-transversally elliptic symbol 
on T*A: we have Char(c K ) n T K2 (T*A) = {(1,0)}. We will now compute the 
equivariant index of c K . 

First we consider the homotopy t e [0, 1] — > Jt£ of symplcctic structure on T* A. 
Let c K be the symbol acting on A* (6 x i*) = A*tc- Proposition 12 . 61 shows that the 
symbols c K and c K define the same class in Kk 1 xk 2 (Tk 2 {T* K)). 

The projection 7r : T*A — > 6* corresponds to the quotient map relative to the 
free action of A 2 . At the level of A-groups we get an isomorphism 

^* : K KlxK2 (T K2 (T*K)) -» K Kl (TF). 

Atiyah [1] proves that 

Index^ 2 (a) = £ Index^ (^(a ® V^)) ® {V^-f 

for any class a £ Kkxxk 2 (^k 2 (!T*K)). In our case the symbol n*(c K ) is equal to 
the Bott symbol Bott(t*), and for any AVmodule £ 2 we have 

7r»(c K ® A 2 ) = Bott(l*) ® £7i 

where E\ is the module £2 with the action of K\. Then 

Q| lX K 2 (T*A) = Index^ 2 (c K ) 

= ^Index^jBott^)®^)®^)* 

since Index^ i (Bott(r)) = 1. 

5.2. The formal quantization of T*(K/H). Let if be a closed connected sub- 
group of A. Look at T*K as a Hamiltonian manifold relatively to the action of 
H x K C A'i x A 2 . The moment map $ = ($h,®k) is defined by : <f>//(a,£) = 
— pr(£) and $k((i, £) = a ' £j where pr : 6* — > (}* is the projection. Note that $ is a 
proper map. 

The cotangent bundle T*(K/H), viewed as A- manifold, is equal to the symplcc- 
tic reduction of T* A relatively to the fAaction: if the kernel of the projection pr 
is denoted rj- 1 , we have 

^(OV-ff = Kx H \ ) ± = T*(K/H). 
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We are here in the setting of section 14.21 The reduction of the H x K proper 
Hamiltonian manifold T*K relatively to the H- action is smooth, then its formal 
quantization is computed as follows 

Q%(T*(K/H))=[Q% xK (T*K)] H = [Q% 1XK2 (T*K)\ HXK ] H 

(5.56) = [L 2 (K)] H 

= L 2 (K/H). 

Here the fact that Q% xK (T*K) is ec l ual to thc restriction of Q| lxX ,(T*A") = 
L 2 (K) to H x K is a consequence of Theorem 11.31 

Let us denoted T*(K/H) the symplcctic reduction at fx G K of the K- 
Hamiltonian manifold T*(K/H). Theorem II .41 together with (| 1 .6|) gives 

Corollary 5.1. For any /i G K , we have 



Q 



T*(K/H) 



= dim 



V K 



where \V^] H is the subspace of H -invariant vector. 

5.3. The formal quantization of T*(K/H) relatively to the action of G. 

Let G be a closed connected subgroup of K . We look at the hamiltonian action of 
G on T*(K/H). Let <I>g : T*(K/H) — > g* be the moment map. We consider also 
the restriction of the K- module L 2 (K/H) to G. 
We have 

Proposition 5.2. The following statements are equivalent 

(1) The moment map $g : T*(K/H) —* g* is proper. 

(2) 3>q (0) is equal to the zero section. 

(3) k ■ Q + t) = t, for any k E K. 

(4) + & = t 

(5) G acts transitively on K/H. 

(6) L 2 {K/H)] G ~C 

(7) L (K/H)\g is an admissible G -representation. 

Proof. (1) => (7) is a consequence of Theorem 11.31 Let us prove that (7) ==> (6). 
Suppose now that 

l 2 (k/h)\ g = J2[vf] H ®(vfy\ G 

fiEK 

is an admissible G-representation. It means that for any A G G the set 

Ax := {m G £ | ^ {0} and [(Vjf)* ® (Vf )*| G ] G ^ {0}} 

is finite. Then the vector space L 2 (K/H)] G is equal to the finite dimensional vector 
space E MeJ 4 [ y /f] ff ® l( V i?)*] G - :t is not difficult to check that if (i G A , then 
fc/i G Aq for fc >> 1. Finally the fact that Aq is finite implies that Ao is reduced to 
/i = 0. Hence the only G-invariant functions on K/H are the scalars. 

(6) •<=>• (5) •<==>• (4) <^=> (3) is a general fact concerning smooth actions of a 
compact connected Lie group G on a compact connected manifold M. The manifold 
M does not have G-invariant functions which are not scalar if and only if the action 
of G on M is transitive. And given a point m G M, the orbit G • m is all of M if 
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and only if tangent spaces T m (G • m) and T m M are equal. If we take m = k 1 in 
M = K/H, the condition T m (G • m) = T m M is equivalent to k ■ fj+ f) = 4. 

Let us check (3) => (2). Let G A'x^r/ = T* (If/if) .We have $ G ([jfe, £]) = 
if and only if • £ G JJ . Hence the vector £ belongs to 

k- 1 -S X f)t) ± = (k- 1 -g + f)) x . 

Hence condition (3) imposes that £ = 0. 

(2) ^==> (1) comes from the fact that $g is a homogeneous map of degree one 
between the vector bundle T*(K/H) and the vector space g*. □ 

Suppose now that the cotangent bundle T*(K/H) is a proper Hamiltonian G- 
manifold. Let us denoted \T*{K/H)] )ltG the (compact) symplectic reduction at 
H G G of the G-Hamiltonian manifold T*(K/H). Then, 

Corollary 5.3. The multiplicity ofV^ in L 2 (K/H) is equal to the quantization of 
the reduced space [T*(K/H)]^ ,g. 

Proof. Using Theorem II .31 equality (|5 . 56[) gives then 

Qc°°(T*(K/H)) = Q-°°(T*(K/H))\ G 
= L 2 (K/H)\ G . 

In other words, the multiplicity of in L 2 (K/H) is equal to the quantization of 
the reduced space [T*(K/H)]^ G - □ 
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